hbase 单机连接hadoop

您所在的位置:网站首页 hadoop jps只有jps hbase 单机连接hadoop

hbase 单机连接hadoop

2022-12-15 09:05| 来源: 网络整理| 查看: 265

auth  修改为:auth

hive-site.xml 文件:

首先修改配置项

hive.metastore.schema.verification

false(这个值改成false)

文件最后加上几行:

hive.aux.jars.path

file:///home/hadoop/hive-0.12.0/lib/hive-hbase-handler-0.12.0.jar,file:///home/hadoop/hive-0.12.0/lib/hbase-0.94.18.jar,file:///home/hadoop/hive-0.12.0/lib/zookeeper-3.4.5.jar

注意写法,目录形式是file:///,一定要一字不差,然后将对应的jar包从hbase目录下拷贝到相应位置,如果hive下已有对应jar包,建议删除后使用hbase目录下的。

二、启动并测试hive(以下内容来自http://blog.csdn.net/luckymelina/article/details/16707801)

进入hive目录下,敲命令:

bin/hive -hiveconf hbase.master=master:60000

a)       创建hbase识别的表

CREATE TABLE hbase_table_1(key int, value string) STORED

BY'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH

SERDEPROPERTIES("hbase.columns.mapping" = ":key,cf1:val")TBLPROPERTIES ("hbase.table.name" = "xyz");

b)       新建hive的数据表

create table pokes(foo int,bar string)row format delimited fields terminated by ',';

c)       批量导入数据

load data local inpath '/home/hadoop/1.txt' overwrite into table pokes;

1.txt文件的内容为

1,hello

2,pear

3,world

d)      使用sql导入hbase_table_1

insert overwrite table hbase_table_1 select * from pokes;

这一步如果正常说明hbase与hive已做到了正常连接,输出日志:

Total MapReduce jobs = 1

Launching Job 1 out of 1

Number of reduce tasks is set to 0 since there's no reduce operator

Starting Job = job_201404191501_0011, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201404191501_0011

Kill Command = /home/hadoop/hadoop/libexec/../bin/hadoop job  -kill job_201404191501_0011

Hadoop job information for Stage-0: number of mappers: 1; number of reducers: 0

2014-04-19 21:14:21,307 Stage-0 map = 0%,  reduce = 0%

2014-04-19 21:14:35,384 Stage-0 map = 100%,  reduce = 0%, Cumulative CPU 2.39 sec

2014-04-19 21:14:36,391 Stage-0 map = 100%,  reduce = 0%, Cumulative CPU 2.39 sec

2014-04-19 21:14:37,398 Stage-0 map = 100%,  reduce = 0%, Cumulative CPU 2.39 sec

2014-04-19 21:14:38,406 Stage-0 map = 100%,  reduce = 0%, Cumulative CPU 2.39 sec

2014-04-19 21:14:39,416 Stage-0 map = 100%,  reduce = 100%, Cumulative CPU 2.39 sec

MapReduce Total cumulative CPU time: 2 seconds 390 msec

Ended Job = job_201404191501_0011

MapReduce Jobs Launched:

Job 0: Map: 1   Cumulative CPU: 2.39 sec   HDFS Read: 232 HDFS Write: 0 SUCCESS

Total MapReduce CPU Time Spent: 2 seconds 390 msec

OK

Time taken: 57.471 seconds

e)       查看数据

hive>select * from hbase_table_1;

OK

1 hello

2 pear

3 world

说明相应数据已成功的从Hive导入到HBase中



【本文地址】


今日新闻


推荐新闻


CopyRight 2018-2019 办公设备维修网 版权所有 豫ICP备15022753号-3